Anatomy-Guided Weakly-Supervised Abnormality Localization in Chest X-rays

Creating a large-scale dataset of abnormality annotation on medical images is labor-intensive and costly task. Leveraging weak supervision from readily available data such as radiology reports can compensate lack for anomaly detection methods. However, most the current methods only use image-level pathological observations, failing to utilize relevant anatomy mentions in reports. Furthermore, Natural Language Processing (NLP)-mined labels are noisy due label sparsity linguistic ambiguity. We propose an Anatomy-Guided chest X-ray Network (AGXNet) address these issues annotation. Our framework consists cascade two networks, one responsible identifying anatomical abnormalities second observations. The critical component our anatomy-guided attention module that aids downstream observation network focusing regions generated by network. Positive Unlabeled (PU) learning account fact mention does not necessarily mean negative label. quantitative qualitative results MIMIC-CXR demonstrate effectiveness AGXNet disease localization. Experiments NIH Chest show learned feature representations transferable achieve state-of-the-art performances classification competitive localization results. code at https://github.com/batmanlab/AGXNet .